Skip to content

feat: separate manually added bookmarks from RSS subscriptions in UI and rules#2549

Closed
VedantMadane wants to merge 5 commits intokarakeep-app:mainfrom
VedantMadane:feat/separate-manual-and-rss-bookmarks
Closed

feat: separate manually added bookmarks from RSS subscriptions in UI and rules#2549
VedantMadane wants to merge 5 commits intokarakeep-app:mainfrom
VedantMadane:feat/separate-manual-and-rss-bookmarks

Conversation

@VedantMadane
Copy link

Fixes #2526.

Summary

This PR adds the ability to distinguish between manually added bookmarks and those received via RSS subscriptions, both visually and through automated rules.

Key Changes:

  • Rule Engine: Added a new \�ookmarkSourceIs\ condition, allowing users to automate actions based on where a bookmark came from (e.g., tagging all RSS items).
  • UI Indicators: Added a \SourceIndicator\ to \BookmarkLayoutAdaptingCard. RSS bookmarks now show an orange RSS icon, while manual bookmarks (Web, Extension, Mobile, CLI) show a blue pointer icon.
  • Search Support: The search parser already supported \source:\ filters (e.g., \source:rss), which now work seamlessly with these improvements.
  • Localization: Added new translation strings for bookmark sources in \ ranslation.json.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 2, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 72933cde-3503-4700-80c8-f1786fa28f8f

📥 Commits

Reviewing files that changed from the base of the PR and between 4e0cd2e and f30e4f5.

📒 Files selected for processing (1)
  • apps/web/components/dashboard/bookmarks/BookmarkLayoutAdaptingCard.tsx
🚧 Files skipped from review as they are similar to previous changes (1)
  • apps/web/components/dashboard/bookmarks/BookmarkLayoutAdaptingCard.tsx

Walkthrough

Adds bookmark source awareness across the app: UI shows source indicators, the rule system gains a new bookmarkSourceIs condition with types and translations, and the crawler worker adds richer asset tracking and cleanup on failures.

Changes

Cohort / File(s) Summary
Bookmark UI
apps/web/components/dashboard/bookmarks/BookmarkLayoutAdaptingCard.tsx
Adds internal SourceIndicator component; renders RSS or manual/source badges in List, Grid, and Compact layouts.
Rule Builder UI
apps/web/components/dashboard/rules/RuleEngineConditionBuilder.tsx
Adds bookmarkSourceIs condition option, UI Select for source values, and Database icon rendering for the new condition.
Translations
apps/web/lib/i18n/locales/en/translation.json
Adds bookmark_sources labels and bookmark_source_is condition translation entries.
Shared Rule Types
packages/shared/types/rules.ts
Introduces zBookmarkSourceIsCondition, adds it to the rule condition union and validator switch.
Rule Engine
packages/trpc/lib/ruleEngine.ts
Implements bookmarkSourceIs branch in doesBookmarkMatchConditions to compare bookmark.source to condition.source.
Crawler Worker
apps/workers/workers/crawlerWorker.ts
Refactors asset handling: collects new asset IDs, enriches asset metadata, inserts bookmarkAssets, converts LINK→ASSET, enqueues preprocessing, and ensures cleanup of partially created assets on errors.

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 11.11% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The PR title clearly summarizes the main objective: adding separation between manually added bookmarks and RSS subscriptions in both UI and rules.
Description check ✅ Passed The PR description is well-related to the changeset, explaining the purpose and key changes including rule engine, UI indicators, search support, and localization.
Linked Issues check ✅ Passed The PR successfully addresses #2526 by implementing UI indicators to distinguish bookmark sources and adding rule engine support for source-based automation, meeting all stated objectives.
Out of Scope Changes check ✅ Passed All changes are directly scoped to the linked issue objectives: UI indicators, rule engine conditions, and localization support for distinguishing bookmark sources.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@apps/web/components/dashboard/bookmarks/BookmarkLayoutAdaptingCard.tsx`:
- Around line 99-105: The tooltip always uses t("common.bookmark_sources.web")
for all manualSources, causing extension/mobile/cli to be labeled "Web"; update
the title logic in BookmarkLayoutAdaptingCard so it derives the correct
translation key from bookmark.source (e.g. map or compute
`common.bookmark_sources.${bookmark.source}` or use a small switch) instead of
hardcoding "web", and fall back to a generic label if the source key is missing;
adjust the title prop on the div where manualSources is checked to use that
dynamic translation.
- Around line 414-416: SourceIndicator is positioned in the top-right where
owner and hover action overlays also render, causing overlap; in
BookmarkLayoutAdaptingCard change its wrapper positioning (the div containing
<SourceIndicator bookmark={bookmark} />) to a non-conflicting corner or offset
(for example replace "absolute right-2 top-2 z-40" with "absolute left-2 top-2
z-40" or "absolute right-8 top-2 z-40") so it no longer overlaps owner/hover
indicators, and run/update any visual/snapshot tests affected by the layout
change.

In `@apps/workers/workers/crawlerWorker.ts`:
- Around line 1559-1619: The catch block unconditionally deletes
downloadedAssetId even if the DB transaction committed and the asset is now
live; modify handleAsAssetBookmark so cleanup only runs when the transaction did
not commit: after calling downloadAndStoreFile set downloadedAssetId, introduce
a boolean like transactionCommitted (default false), set it to true at the end
of the db.transaction callback (after updateAsset/insert/update/delete), then
call AssetPreprocessingQueue.enqueue; in the catch, only call
silentDeleteAsset(userId, downloadedAssetId) if !transactionCommitted (i.e., the
DB changes never committed). Reference symbols: downloadedAssetId,
downloadAndStoreFile, db.transaction, AssetPreprocessingQueue.enqueue,
silentDeleteAsset.
- Around line 1836-1848: The code races storeScreenshot/storePdf against
abortPromise but doesn’t cancel the underlying storage calls, which can complete
and create assets that never get tracked; modify the storage flow so storage is
abortable and/or cleaned up: update storeScreenshot and storePdf to accept an
AbortSignal (or return a handle) and have them abort any in-flight upload when
abortSignal is triggered, or after the storage promise resolves check
abortSignal.aborted and if aborted call the corresponding cleanup/delete
function for the returned asset before returning/adding to newAssetIds; ensure
you only push screenshotAssetInfo.assetId / pdfAssetInfo.assetId into
newAssetIds after confirming the operation wasn’t aborted (use
abortSignal.throwIfAborted or signal.aborted) and wire abortSignal into the
Promise.race path so no orphaned assets remain.

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 85e63ae and f188670.

📒 Files selected for processing (6)
  • apps/web/components/dashboard/bookmarks/BookmarkLayoutAdaptingCard.tsx
  • apps/web/components/dashboard/rules/RuleEngineConditionBuilder.tsx
  • apps/web/lib/i18n/locales/en/translation.json
  • apps/workers/workers/crawlerWorker.ts
  • packages/shared/types/rules.ts
  • packages/trpc/lib/ruleEngine.ts

Comment on lines +1559 to +1619
let downloadedAssetId: string | undefined;
try {
const downloaded = await downloadAndStoreFile(
url,
userId,
jobId,
assetType,
abortSignal,
);
if (!downloaded) {
return;
}
downloadedAssetId = downloaded.assetId;
const fileName = path.basename(new URL(url).pathname);
await db.transaction(async (trx) => {
await updateAsset(
undefined,
{
id: downloaded.assetId,
bookmarkId,
userId,
assetType: AssetTypes.BOOKMARK_ASSET,
contentType: downloaded.contentType,
size: downloaded.size,
fileName,
},
trx,
);
await trx.insert(bookmarkAssets).values({
id: bookmarkId,
assetType,
assetId: downloaded.assetId,
content: null,
fileName,
sourceUrl: url,
});
// Switch the type of the bookmark from LINK to ASSET
await trx
.update(bookmarks)
.set({ type: BookmarkTypes.ASSET })
.where(eq(bookmarks.id, bookmarkId));
await trx.delete(bookmarkLinks).where(eq(bookmarkLinks.id, bookmarkId));
});
await AssetPreprocessingQueue.enqueue(
{
id: downloaded.assetId,
bookmarkId,
userId,
assetType: AssetTypes.BOOKMARK_ASSET,
contentType: downloaded.contentType,
size: downloaded.size,
fileName,
fixMode: false,
},
{
groupId: userId,
},
trx,
);
await trx.insert(bookmarkAssets).values({
id: bookmarkId,
assetType,
assetId: downloaded.assetId,
content: null,
fileName,
sourceUrl: url,
});
// Switch the type of the bookmark from LINK to ASSET
await trx
.update(bookmarks)
.set({ type: BookmarkTypes.ASSET })
.where(eq(bookmarks.id, bookmarkId));
await trx.delete(bookmarkLinks).where(eq(bookmarkLinks.id, bookmarkId));
});
await AssetPreprocessingQueue.enqueue(
{
bookmarkId,
fixMode: false,
},
{
groupId: userId,
},
);
} catch (error) {
if (downloadedAssetId) {
logger.error(
`[Crawler][${jobId}] handleAsAssetBookmark encountered an error, cleaning up new asset ${downloadedAssetId}: ${error}`,
);
await silentDeleteAsset(userId, downloadedAssetId);
}
throw error;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Post-commit enqueue failure can delete a committed asset.

downloadedAssetId cleanup in the catch block runs even when the DB transaction already committed (Line 1573-1601). If enqueue fails afterward, this deletes live data and leaves the converted bookmark inconsistent.

🔧 Suggested fix
-      let downloadedAssetId: string | undefined;
+      let downloadedAssetId: string | undefined;
+      let conversionCommitted = false;
       try {
         const downloaded = await downloadAndStoreFile(
           url,
           userId,
           jobId,
           assetType,
           abortSignal,
         );
         if (!downloaded) {
           return;
         }
         downloadedAssetId = downloaded.assetId;
         const fileName = path.basename(new URL(url).pathname);
         await db.transaction(async (trx) => {
           await updateAsset(
             undefined,
             {
               id: downloaded.assetId,
               bookmarkId,
               userId,
               assetType: AssetTypes.BOOKMARK_ASSET,
               contentType: downloaded.contentType,
               size: downloaded.size,
               fileName,
             },
             trx,
           );
           await trx.insert(bookmarkAssets).values({
             id: bookmarkId,
             assetType,
             assetId: downloaded.assetId,
             content: null,
             fileName,
             sourceUrl: url,
           });
           await trx
             .update(bookmarks)
             .set({ type: BookmarkTypes.ASSET })
             .where(eq(bookmarks.id, bookmarkId));
           await trx.delete(bookmarkLinks).where(eq(bookmarkLinks.id, bookmarkId));
         });
+        conversionCommitted = true;
-        await AssetPreprocessingQueue.enqueue(
-          {
-            bookmarkId,
-            fixMode: false,
-          },
-          {
-            groupId: userId,
-          },
-        );
+        await AssetPreprocessingQueue.enqueue(
+          { bookmarkId, fixMode: false },
+          { groupId: userId },
+        );
       } catch (error) {
-        if (downloadedAssetId) {
+        if (!conversionCommitted && downloadedAssetId) {
           logger.error(
             `[Crawler][${jobId}] handleAsAssetBookmark encountered an error, cleaning up new asset ${downloadedAssetId}: ${error}`,
           );
           await silentDeleteAsset(userId, downloadedAssetId);
         }
         throw error;
       }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let downloadedAssetId: string | undefined;
try {
const downloaded = await downloadAndStoreFile(
url,
userId,
jobId,
assetType,
abortSignal,
);
if (!downloaded) {
return;
}
downloadedAssetId = downloaded.assetId;
const fileName = path.basename(new URL(url).pathname);
await db.transaction(async (trx) => {
await updateAsset(
undefined,
{
id: downloaded.assetId,
bookmarkId,
userId,
assetType: AssetTypes.BOOKMARK_ASSET,
contentType: downloaded.contentType,
size: downloaded.size,
fileName,
},
trx,
);
await trx.insert(bookmarkAssets).values({
id: bookmarkId,
assetType,
assetId: downloaded.assetId,
content: null,
fileName,
sourceUrl: url,
});
// Switch the type of the bookmark from LINK to ASSET
await trx
.update(bookmarks)
.set({ type: BookmarkTypes.ASSET })
.where(eq(bookmarks.id, bookmarkId));
await trx.delete(bookmarkLinks).where(eq(bookmarkLinks.id, bookmarkId));
});
await AssetPreprocessingQueue.enqueue(
{
id: downloaded.assetId,
bookmarkId,
userId,
assetType: AssetTypes.BOOKMARK_ASSET,
contentType: downloaded.contentType,
size: downloaded.size,
fileName,
fixMode: false,
},
{
groupId: userId,
},
trx,
);
await trx.insert(bookmarkAssets).values({
id: bookmarkId,
assetType,
assetId: downloaded.assetId,
content: null,
fileName,
sourceUrl: url,
});
// Switch the type of the bookmark from LINK to ASSET
await trx
.update(bookmarks)
.set({ type: BookmarkTypes.ASSET })
.where(eq(bookmarks.id, bookmarkId));
await trx.delete(bookmarkLinks).where(eq(bookmarkLinks.id, bookmarkId));
});
await AssetPreprocessingQueue.enqueue(
{
bookmarkId,
fixMode: false,
},
{
groupId: userId,
},
);
} catch (error) {
if (downloadedAssetId) {
logger.error(
`[Crawler][${jobId}] handleAsAssetBookmark encountered an error, cleaning up new asset ${downloadedAssetId}: ${error}`,
);
await silentDeleteAsset(userId, downloadedAssetId);
}
throw error;
}
let downloadedAssetId: string | undefined;
let conversionCommitted = false;
try {
const downloaded = await downloadAndStoreFile(
url,
userId,
jobId,
assetType,
abortSignal,
);
if (!downloaded) {
return;
}
downloadedAssetId = downloaded.assetId;
const fileName = path.basename(new URL(url).pathname);
await db.transaction(async (trx) => {
await updateAsset(
undefined,
{
id: downloaded.assetId,
bookmarkId,
userId,
assetType: AssetTypes.BOOKMARK_ASSET,
contentType: downloaded.contentType,
size: downloaded.size,
fileName,
},
trx,
);
await trx.insert(bookmarkAssets).values({
id: bookmarkId,
assetType,
assetId: downloaded.assetId,
content: null,
fileName,
sourceUrl: url,
});
// Switch the type of the bookmark from LINK to ASSET
await trx
.update(bookmarks)
.set({ type: BookmarkTypes.ASSET })
.where(eq(bookmarks.id, bookmarkId));
await trx.delete(bookmarkLinks).where(eq(bookmarkLinks.id, bookmarkId));
});
conversionCommitted = true;
await AssetPreprocessingQueue.enqueue(
{ bookmarkId, fixMode: false },
{ groupId: userId },
);
} catch (error) {
if (!conversionCommitted && downloadedAssetId) {
logger.error(
`[Crawler][${jobId}] handleAsAssetBookmark encountered an error, cleaning up new asset ${downloadedAssetId}: ${error}`,
);
await silentDeleteAsset(userId, downloadedAssetId);
}
throw error;
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/workers/workers/crawlerWorker.ts` around lines 1559 - 1619, The catch
block unconditionally deletes downloadedAssetId even if the DB transaction
committed and the asset is now live; modify handleAsAssetBookmark so cleanup
only runs when the transaction did not commit: after calling
downloadAndStoreFile set downloadedAssetId, introduce a boolean like
transactionCommitted (default false), set it to true at the end of the
db.transaction callback (after updateAsset/insert/update/delete), then call
AssetPreprocessingQueue.enqueue; in the catch, only call
silentDeleteAsset(userId, downloadedAssetId) if !transactionCommitted (i.e., the
DB changes never committed). Reference symbols: downloadedAssetId,
downloadAndStoreFile, db.transaction, AssetPreprocessingQueue.enqueue,
silentDeleteAsset.

Comment on lines +1836 to +1848
const screenshotAssetInfo = await Promise.race([
storeScreenshot(screenshot, userId, jobId),
abortPromise(abortSignal),
]);
if (screenshotAssetInfo) {
await updateAsset(
oldScreenshotAssetId,
{
id: screenshotAssetInfo.assetId,
bookmarkId,
userId,
assetType: AssetTypes.LINK_SCREENSHOT,
contentType: screenshotAssetInfo.contentType,
size: screenshotAssetInfo.size,
fileName: screenshotAssetInfo.fileName,
},
txn,
);
assetDeletionTasks.push(
silentDeleteAsset(userId, oldScreenshotAssetId),
);
newAssetIds.push(screenshotAssetInfo.assetId);
}
abortSignal.throwIfAborted();

const pdfAssetInfo = await Promise.race([
storePdf(pdf, userId, jobId),
abortPromise(abortSignal),
]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Abort racing can leave untracked assets behind.

Promise.race here does not cancel storeScreenshot/storePdf. If abort wins first, the storage promise can still complete and persist assets whose IDs never get added to newAssetIds.

🔧 Suggested fix
-        const screenshotAssetInfo = await Promise.race([
-          storeScreenshot(screenshot, userId, jobId),
-          abortPromise(abortSignal),
-        ]);
+        const screenshotAssetInfo = await storeScreenshot(
+          screenshot,
+          userId,
+          jobId,
+        );
+        abortSignal.throwIfAborted();
         if (screenshotAssetInfo) {
           newAssetIds.push(screenshotAssetInfo.assetId);
         }
-        abortSignal.throwIfAborted();
 
-        const pdfAssetInfo = await Promise.race([
-          storePdf(pdf, userId, jobId),
-          abortPromise(abortSignal),
-        ]);
+        const pdfAssetInfo = await storePdf(pdf, userId, jobId);
+        abortSignal.throwIfAborted();
         if (pdfAssetInfo) {
           newAssetIds.push(pdfAssetInfo.assetId);
         }
-        abortSignal.throwIfAborted();
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/workers/workers/crawlerWorker.ts` around lines 1836 - 1848, The code
races storeScreenshot/storePdf against abortPromise but doesn’t cancel the
underlying storage calls, which can complete and create assets that never get
tracked; modify the storage flow so storage is abortable and/or cleaned up:
update storeScreenshot and storePdf to accept an AbortSignal (or return a
handle) and have them abort any in-flight upload when abortSignal is triggered,
or after the storage promise resolves check abortSignal.aborted and if aborted
call the corresponding cleanup/delete function for the returned asset before
returning/adding to newAssetIds; ensure you only push
screenshotAssetInfo.assetId / pdfAssetInfo.assetId into newAssetIds after
confirming the operation wasn’t aborted (use abortSignal.throwIfAborted or
signal.aborted) and wire abortSignal into the Promise.race path so no orphaned
assets remain.

@greptile-apps
Copy link

greptile-apps bot commented Mar 2, 2026

Greptile Summary

This PR adds the ability to distinguish bookmark sources (RSS vs. manual) both visually (new SourceIndicator icons on bookmark cards) and in the rule engine (new bookmarkSourceIs condition). The feature is well-integrated across the shared type schema, tRPC rule engine, UI condition builder, and i18n strings.

Key issues identified:

  • JSON formatting artifact (translation.json line 384): "and" and "or" were accidentally collapsed onto a single line, likely a merge artifact.
  • newAssetIds not cleared post-transaction (crawlerWorker.ts): After the DB transaction that commits new asset references succeeds, newAssetIds is never reset. If an error occurs after the transaction (e.g., rejection from silentDeleteAsset), the catch block will delete assets that are already live in the database, creating dangling references.

The core bookmark-source feature (types, rule engine, UI indicators) is correctly implemented and low-risk.

Confidence Score: 2/5

  • The PR adds the bookmark-source feature correctly, but contains a logic bug in crawlerWorker.ts that could silently delete committed assets if an error occurs during old asset cleanup.
  • The core bookmark-source feature (types, rule engine, UI indicators) is correctly implemented and low-risk (would be 5/5 in isolation). However, the newAssetIds logic bug in crawlerWorker.ts is a genuine correctness issue: if Promise.all(assetDeletionTasks) throws after the transaction commits, the catch handler will delete assets that are already live in the database. This could silently corrupt asset references in an edge case. Additionally, there is a formatting artifact in translation.json. These issues should be fixed before merging.
  • apps/workers/workers/crawlerWorker.ts requires the newAssetIds clearing fix before merge. apps/web/lib/i18n/locales/en/translation.json requires the formatting fix.

Last reviewed commit: f30e4f5

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6 files reviewed, 3 comments

Edit Code Review Agent Settings | Greptile

VedantMadane and others added 3 commits March 2, 2026 13:57
….tsx

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…Card.tsx

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
@MohamedBassem
Copy link
Collaborator

@greptile review

"and": "All of the following are true",
"or": "Any of the following are true"
"bookmark_source_is": "Bookmark Source Is",
"and": "All of the following are true", "or": "Any of the following are true"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Formatting error: two entries merged onto one line.

A newline was lost between "and" and "or", leaving both JSON entries on the same line. While technically valid JSON, this is clearly unintentional and inconsistent with the rest of the file's formatting.

Suggested change
"and": "All of the following are true", "or": "Any of the following are true"
"and": "All of the following are true",
"or": "Any of the following are true"
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/web/lib/i18n/locales/en/translation.json
Line: 384

Comment:
Formatting error: two entries merged onto one line.

A newline was lost between `"and"` and `"or"`, leaving both JSON entries on the same line. While technically valid JSON, this is clearly unintentional and inconsistent with the rest of the file's formatting.

```suggestion
        "and": "All of the following are true",
        "or": "Any of the following are true"
```

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +1968 to +1971
});

// Delete the old assets if any
await Promise.all(assetDeletionTasks);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newAssetIds not cleared after transaction commits, risking deletion of live assets

Once the db.transaction(...) at line 1893 completes successfully, all entries in newAssetIds are now referenced by the database. However, newAssetIds is never cleared after the transaction. If the subsequent await Promise.all(assetDeletionTasks) throws (e.g., if silentDeleteAsset rejects), the catch block at line 2020 will iterate over newAssetIds and call silentDeleteAsset on every one of them — deleting assets that are already committed and referenced in the DB, leaving dangling references.

The fix is to clear newAssetIds immediately after the transaction succeeds, so the catch handler only ever deletes uncommitted assets:

Suggested change
});
// Delete the old assets if any
await Promise.all(assetDeletionTasks);
});
newAssetIds.length = 0; // Assets are now committed; don't clean them up on error
// Delete the old assets if any
await Promise.all(assetDeletionTasks);
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/workers/workers/crawlerWorker.ts
Line: 1968-1971

Comment:
`newAssetIds` not cleared after transaction commits, risking deletion of live assets

Once the `db.transaction(...)` at line 1893 completes successfully, all entries in `newAssetIds` are now referenced by the database. However, `newAssetIds` is never cleared after the transaction. If the subsequent `await Promise.all(assetDeletionTasks)` throws (e.g., if `silentDeleteAsset` rejects), the catch block at line 2020 will iterate over `newAssetIds` and call `silentDeleteAsset` on every one of them — deleting assets that are already committed and referenced in the DB, leaving dangling references.

The fix is to clear `newAssetIds` immediately after the transaction succeeds, so the catch handler only ever deletes uncommitted assets:

```suggestion
        });
        newAssetIds.length = 0; // Assets are now committed; don't clean them up on error

        // Delete the old assets if any
        await Promise.all(assetDeletionTasks);
```

How can I resolve this? If you propose a fix, please make it concise.

@MohamedBassem
Copy link
Collaborator

Thanks for the PR! it seem that this PR is mixing some content from the other PR about asset cleanup? Mind decoupling them so that we can merge them separately ?

@MohamedBassem
Copy link
Collaborator

Also, it seems that the main improvement was also implemented in #2547. So closing this PR for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Separating manually added bookmarks and those received from RSS subscriptions

2 participants